{
 "cells": [
  {
   "cell_type": "raw",
   "id": "023635f2-71cf-43f2-a2e2-a7b4ced30a74",
   "metadata": {},
   "source": [
    "---\n",
    "sidebar_position: 2\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "86fc5bb2-017f-434e-8cd6-53ab214a5604",
   "metadata": {},
   "source": [
    "# Add chat history\n",
    "\n",
    "In many Q&A applications we want to allow the user to have a back-and-forth conversation, meaning the application needs some sort of \"memory\" of past questions and answers, and some logic for incorporating those into its current thinking.\n",
    "\n",
    "In this guide we focus on **adding logic for incorporating historical messages, and NOT on chat history management.** Chat history management is [covered here](/docs/expression_language/how_to/message_history).\n",
    "\n",
    "We'll work off of the Q&A app we built over the [LLM Powered Autonomous Agents](https://lilianweng.github.io/posts/2023-06-23-agent/) blog post by Lilian Weng in the [Quickstart](/docs/use_cases/question_answering/quickstart). We'll need to update two things about our existing app:\n",
    "\n",
    "1. **Prompt**: Update our prompt to support historical messages as an input.\n",
    "2. **Contextualizing questions**: Add a sub-chain that takes the latest user question and reformulates it in the context of the chat history. This is needed in case the latest question references some context from past messages. For example, if a user asks a follow-up question like \"Can you elaborate on the second point?\", this cannot be understood without the context of the previous message. Therefore we can't effectively perform retrieval with a question like this."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "487d8d79-5ee9-4aa4-9fdf-cd5f4303e099",
   "metadata": {},
   "source": [
    "## Setup\n",
    "\n",
    "### Dependencies\n",
    "\n",
    "We'll use an OpenAI chat model and embeddings and a Chroma vector store in this walkthrough, but everything shown here works with any [ChatModel](/docs/modules/model_io/chat/) or [LLM](/docs/modules/model_io/llms/), [Embeddings](/docs/modules/data_connection/text_embedding/), and [VectorStore](/docs/modules/data_connection/vectorstores/) or [Retriever](/docs/modules/data_connection/retrievers/). \n",
    "\n",
    "We'll use the following packages:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "28d272cd-4e31-40aa-bbb4-0be0a1f49a14",
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install --upgrade --quiet  langchain langchain-community langchainhub langchain-openai chromadb bs4"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "51ef48de-70b6-4f43-8e0b-ab9b84c9c02a",
   "metadata": {},
   "source": [
    "We need to set environment variable `OPENAI_API_KEY`, which can be done directly or loaded from a `.env` file like so:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "143787ca-d8e6-4dc9-8281-4374f4d71720",
   "metadata": {},
   "outputs": [],
   "source": [
    "import getpass\n",
    "import os\n",
    "\n",
    "os.environ[\"OPENAI_API_KEY\"] = getpass.getpass()\n",
    "\n",
    "# import dotenv\n",
    "\n",
    "# dotenv.load_dotenv()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1665e740-ce01-4f09-b9ed-516db0bd326f",
   "metadata": {},
   "source": [
    "### LangSmith\n",
    "\n",
    "Many of the applications you build with LangChain will contain multiple steps with multiple invocations of LLM calls. As these applications get more and more complex, it becomes crucial to be able to inspect what exactly is going on inside your chain or agent. The best way to do this is with [LangSmith](https://smith.langchain.com).\n",
    "\n",
    "Note that LangSmith is not needed, but it is helpful. If you do want to use LangSmith, after you sign up at the link above, make sure to set your environment variables to start logging traces:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "07411adb-3722-4f65-ab7f-8f6f57663d11",
   "metadata": {},
   "outputs": [],
   "source": [
    "os.environ[\"LANGCHAIN_TRACING_V2\"] = \"true\"\n",
    "os.environ[\"LANGCHAIN_API_KEY\"] = getpass.getpass()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fa6ba684-26cf-4860-904e-a4d51380c134",
   "metadata": {},
   "source": [
    "## Chain without chat history\n",
    "\n",
    "Here is the Q&A app we built over the [LLM Powered Autonomous Agents](https://lilianweng.github.io/posts/2023-06-23-agent/) blog post by Lilian Weng in the [Quickstart](/docs/use_cases/question_answering/quickstart):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "d8a913b1-0eea-442a-8a64-ec73333f104b",
   "metadata": {},
   "outputs": [],
   "source": [
    "import bs4\n",
    "from langchain import hub\n",
    "from langchain_community.document_loaders import WebBaseLoader\n",
    "from langchain_community.vectorstores import Chroma\n",
    "from langchain_core.output_parsers import StrOutputParser\n",
    "from langchain_core.runnables import RunnablePassthrough\n",
    "from langchain_openai import ChatOpenAI, OpenAIEmbeddings\n",
    "from langchain_text_splitters import RecursiveCharacterTextSplitter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "820244ae-74b4-4593-b392-822979dd91b8",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load, chunk and index the contents of the blog.\n",
    "loader = WebBaseLoader(\n",
    "    web_paths=(\"https://lilianweng.github.io/posts/2023-06-23-agent/\",),\n",
    "    bs_kwargs=dict(\n",
    "        parse_only=bs4.SoupStrainer(\n",
    "            class_=(\"post-content\", \"post-title\", \"post-header\")\n",
    "        )\n",
    "    ),\n",
    ")\n",
    "docs = loader.load()\n",
    "\n",
    "text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)\n",
    "splits = text_splitter.split_documents(docs)\n",
    "vectorstore = Chroma.from_documents(documents=splits, embedding=OpenAIEmbeddings())\n",
    "\n",
    "# Retrieve and generate using the relevant snippets of the blog.\n",
    "retriever = vectorstore.as_retriever()\n",
    "prompt = hub.pull(\"rlm/rag-prompt\")\n",
    "llm = ChatOpenAI(model_name=\"gpt-3.5-turbo\", temperature=0)\n",
    "\n",
    "\n",
    "def format_docs(docs):\n",
    "    return \"\\n\\n\".join(doc.page_content for doc in docs)\n",
    "\n",
    "\n",
    "rag_chain = (\n",
    "    {\"context\": retriever | format_docs, \"question\": RunnablePassthrough()}\n",
    "    | prompt\n",
    "    | llm\n",
    "    | StrOutputParser()\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "0d3b0f36-7b56-49c0-8e40-a1aa9ebcbf24",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Task decomposition is a technique used to break down complex tasks into smaller and simpler steps. It can be done through prompting techniques like Chain of Thought or Tree of Thoughts, or by using task-specific instructions or human inputs. Task decomposition helps agents plan ahead and manage complicated tasks more effectively.'"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rag_chain.invoke(\"What is Task Decomposition?\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "776ae958-cbdc-4471-8669-c6087436f0b5",
   "metadata": {},
   "source": [
    "## Contextualizing the question\n",
    "\n",
    "First we'll need to define a sub-chain that takes historical messages and the latest user question, and reformulates the question if it makes reference to any information in the historical information.\n",
    "\n",
    "We'll use a prompt that includes a `MessagesPlaceholder` variable under the name \"chat_history\". This allows us to pass in a list of Messages to the prompt using the \"chat_history\" input key, and these messages will be inserted after the system message and before the human message containing the latest question."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "2b685428-8b82-4af1-be4f-7232c5d55b73",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder\n",
    "\n",
    "contextualize_q_system_prompt = \"\"\"Given a chat history and the latest user question \\\n",
    "which might reference context in the chat history, formulate a standalone question \\\n",
    "which can be understood without the chat history. Do NOT answer the question, \\\n",
    "just reformulate it if needed and otherwise return it as is.\"\"\"\n",
    "contextualize_q_prompt = ChatPromptTemplate.from_messages(\n",
    "    [\n",
    "        (\"system\", contextualize_q_system_prompt),\n",
    "        MessagesPlaceholder(variable_name=\"chat_history\"),\n",
    "        (\"human\", \"{question}\"),\n",
    "    ]\n",
    ")\n",
    "contextualize_q_chain = contextualize_q_prompt | llm | StrOutputParser()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "23cbd8d7-7162-4fb0-9e69-67ea4d4603a5",
   "metadata": {},
   "source": [
    "Using this chain we can ask follow-up questions that reference past messages and have them reformulated into standalone questions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "46ee9aa1-16f1-4509-8dae-f8c71f4ad47d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'What is the definition of \"large\" in the context of a language model?'"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain_core.messages import AIMessage, HumanMessage\n",
    "\n",
    "contextualize_q_chain.invoke(\n",
    "    {\n",
    "        \"chat_history\": [\n",
    "            HumanMessage(content=\"What does LLM stand for?\"),\n",
    "            AIMessage(content=\"Large language model\"),\n",
    "        ],\n",
    "        \"question\": \"What is meant by large\",\n",
    "    }\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "42a47168-4a1f-4e39-bd2d-d5b03609a243",
   "metadata": {},
   "source": [
    "## Chain with chat history\n",
    "\n",
    "And now we can build our full QA chain. \n",
    "\n",
    "Notice we add some routing functionality to only run the \"condense question chain\" when our chat history isn't empty. Here we're taking advantage of the fact that if a function in an LCEL chain returns another chain, that chain will itself be invoked."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "66f275f3-ddef-4678-b90d-ee64576878f9",
   "metadata": {},
   "outputs": [],
   "source": [
    "qa_system_prompt = \"\"\"You are an assistant for question-answering tasks. \\\n",
    "Use the following pieces of retrieved context to answer the question. \\\n",
    "If you don't know the answer, just say that you don't know. \\\n",
    "Use three sentences maximum and keep the answer concise.\\\n",
    "\n",
    "{context}\"\"\"\n",
    "qa_prompt = ChatPromptTemplate.from_messages(\n",
    "    [\n",
    "        (\"system\", qa_system_prompt),\n",
    "        MessagesPlaceholder(variable_name=\"chat_history\"),\n",
    "        (\"human\", \"{question}\"),\n",
    "    ]\n",
    ")\n",
    "\n",
    "\n",
    "def contextualized_question(input: dict):\n",
    "    if input.get(\"chat_history\"):\n",
    "        return contextualize_q_chain\n",
    "    else:\n",
    "        return input[\"question\"]\n",
    "\n",
    "\n",
    "rag_chain = (\n",
    "    RunnablePassthrough.assign(\n",
    "        context=contextualized_question | retriever | format_docs\n",
    "    )\n",
    "    | qa_prompt\n",
    "    | llm\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "51fd0e54-5bb4-4a9a-b012-87a18ebe2bef",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AIMessage(content='Common ways of task decomposition include:\\n\\n1. Using Chain of Thought (CoT): CoT is a prompting technique that instructs the model to \"think step by step\" and decompose complex tasks into smaller and simpler steps. This approach utilizes more computation at test-time and sheds light on the model\\'s thinking process.\\n\\n2. Prompting with LLM: Language Model (LLM) can be used to prompt the model with simple instructions like \"Steps for XYZ\" or \"What are the subgoals for achieving XYZ?\" This method guides the model to break down the task into manageable steps.\\n\\n3. Task-specific instructions: For certain tasks, task-specific instructions can be provided to guide the model in decomposing the task. For example, for writing a novel, the instruction \"Write a story outline\" can be given to help the model break down the task into smaller components.\\n\\n4. Human inputs: In some cases, human inputs can be used to assist in task decomposition. Humans can provide insights, expertise, and domain knowledge to help break down complex tasks into smaller subtasks.\\n\\nThese approaches aim to simplify complex tasks and enable more effective problem-solving and planning.')"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "chat_history = []\n",
    "\n",
    "question = \"What is Task Decomposition?\"\n",
    "ai_msg = rag_chain.invoke({\"question\": question, \"chat_history\": chat_history})\n",
    "chat_history.extend([HumanMessage(content=question), ai_msg])\n",
    "\n",
    "second_question = \"What are common ways of doing it?\"\n",
    "rag_chain.invoke({\"question\": second_question, \"chat_history\": chat_history})"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "53263a65-4de2-4dd8-9291-6a8169ab6f1d",
   "metadata": {},
   "source": [
    ":::tip\n",
    "\n",
    "Check out the [LangSmith trace](https://smith.langchain.com/public/b3001782-bb30-476a-886b-12da17ec258f/r) \n",
    "\n",
    ":::"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fdf6c7e0-84f8-4747-b2ae-e84315152bd9",
   "metadata": {},
   "source": [
    "Here we've gone over how to add application logic for incorporating historical outputs, but we're still manually updating the chat history and inserting it into each input. In a real Q&A application we'll want some way of persisting chat history and some way of automatically inserting and updating it.\n",
    "\n",
    "For this we can use:\n",
    "\n",
    "- [BaseChatMessageHistory](/docs/modules/memory/chat_messages/): Store chat history.\n",
    "- [RunnableWithMessageHistory](/docs/expression_language/how_to/message_history): Wrapper for an LCEL chain and a `BaseChatMessageHistory` that handles injecting chat history into inputs and updating it after each invocation.\n",
    "\n",
    "For a detailed walkthrough of how to use these classes together to create a stateful conversational chain, head to the [How to add message history (memory)](/docs/expression_language/how_to/message_history) LCEL page."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1f67a60a-0a31-4315-9cce-19c78d658f6a",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "poetry-venv",
   "language": "python",
   "name": "poetry-venv"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
